{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1c80e2c8-90e4-48cb-a0d3-a849c64bc964",
   "metadata": {},
   "source": [
    "# 多元线性回归\n",
    "跟着油管[GrandmaCan -我阿嬷都会](https://www.youtube.com/watch?v=wm9yR1VspPs)学习的案例。\n",
    "\n",
    "\n",
    "## 数据读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "a0563cdb-7e0c-4540-8bab-f03714758d5c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T18:30:37.728604Z",
     "iopub.status.busy": "2023-05-14T18:30:37.727663Z",
     "iopub.status.idle": "2023-05-14T18:30:37.740832Z",
     "shell.execute_reply": "2023-05-14T18:30:37.739598Z",
     "shell.execute_reply.started": "2023-05-14T18:30:37.728535Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from tqdm.notebook import tqdm\n",
    "# data = pd.read_csv(\"https://raw.githubusercontent.com/GrandmaCan/ML/main/Resgression/Salary_Data2.csv\")\n",
    "# data.to_csv(\"data2.csv\")\n",
    "data = pd.read_csv(\"data2.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ff3fd70e-4e68-43c3-9290-b5c74945cb73",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:07.349469Z",
     "iopub.status.busy": "2023-05-14T17:58:07.348691Z",
     "iopub.status.idle": "2023-05-14T17:58:07.369322Z",
     "shell.execute_reply": "2023-05-14T17:58:07.368423Z",
     "shell.execute_reply.started": "2023-05-14T17:58:07.349429Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>EducationLevel</th>\n",
       "      <th>City</th>\n",
       "      <th>Salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>大學</td>\n",
       "      <td>城市A</td>\n",
       "      <td>45.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>7.8</td>\n",
       "      <td>碩士以上</td>\n",
       "      <td>城市C</td>\n",
       "      <td>80.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>高中以下</td>\n",
       "      <td>城市A</td>\n",
       "      <td>25.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>高中以下</td>\n",
       "      <td>城市A</td>\n",
       "      <td>30.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>10.0</td>\n",
       "      <td>碩士以上</td>\n",
       "      <td>城市B</td>\n",
       "      <td>65.7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  YearsExperience EducationLevel City  Salary\n",
       "0           0              3.0             大學  城市A    45.9\n",
       "1           1              7.8           碩士以上  城市C    80.5\n",
       "2           2              2.3           高中以下  城市A    25.2\n",
       "3           3              5.1           高中以下  城市A    30.4\n",
       "4           4             10.0           碩士以上  城市B    65.7"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "083a87b4-813f-4436-bf3e-16450b5a22d4",
   "metadata": {},
   "source": [
    "## 标签处理\n",
    "显然，教育程度存在递进的关系。我们可以分别给0，1，2的数值进行计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fe85610d-4010-4dd8-b649-1bb31c18d4b7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:09.552535Z",
     "iopub.status.busy": "2023-05-14T17:58:09.551833Z",
     "iopub.status.idle": "2023-05-14T17:58:09.564748Z",
     "shell.execute_reply": "2023-05-14T17:58:09.562948Z",
     "shell.execute_reply.started": "2023-05-14T17:58:09.552473Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 标签处理(label encoding)\n",
    "data[\"EducationLevel\"] = data[\"EducationLevel\"].map({\"高中以下\":0, \"大學\":1, \"碩士以上\":2})\n",
    "# data[\"City\"] = data[\"City\"].map({\"城市A\":1, \"城市B\":2, \"城市C\":3})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f2e0c7bf-b391-438e-bd92-07424772acee",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:11.104758Z",
     "iopub.status.busy": "2023-05-14T17:58:11.104037Z",
     "iopub.status.idle": "2023-05-14T17:58:11.122026Z",
     "shell.execute_reply": "2023-05-14T17:58:11.119998Z",
     "shell.execute_reply.started": "2023-05-14T17:58:11.104694Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>EducationLevel</th>\n",
       "      <th>City</th>\n",
       "      <th>Salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "      <td>城市A</td>\n",
       "      <td>45.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>7.8</td>\n",
       "      <td>2</td>\n",
       "      <td>城市C</td>\n",
       "      <td>80.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0</td>\n",
       "      <td>城市A</td>\n",
       "      <td>25.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>0</td>\n",
       "      <td>城市A</td>\n",
       "      <td>30.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2</td>\n",
       "      <td>城市B</td>\n",
       "      <td>65.7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  YearsExperience  EducationLevel City  Salary\n",
       "0           0              3.0               1  城市A    45.9\n",
       "1           1              7.8               2  城市C    80.5\n",
       "2           2              2.3               0  城市A    25.2\n",
       "3           3              5.1               0  城市A    30.4\n",
       "4           4             10.0               2  城市B    65.7"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b75fa43e-cd6e-4cc9-8bf5-1298143d3fdc",
   "metadata": {},
   "source": [
    "但是城市之间并没有数值关系吧，不同城市怎么排顺序？可以分成三个特征：（one hot encoding）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4b5d11e7-a3d8-4469-bca6-7dfe29079873",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:13.917996Z",
     "iopub.status.busy": "2023-05-14T17:58:13.917287Z",
     "iopub.status.idle": "2023-05-14T17:58:14.564795Z",
     "shell.execute_reply": "2023-05-14T17:58:14.563553Z",
     "shell.execute_reply.started": "2023-05-14T17:58:13.917935Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<36x3 sparse matrix of type '<class 'numpy.float64'>'\n",
       "\twith 36 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "encoder = OneHotEncoder()\n",
    "encoder.fit(data[['City']])\n",
    "city_encoded = encoder.transform(data[[\"City\"]])\n",
    "city_encoded"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f336bbf2-cd1a-46a6-81ca-4482b7bfd654",
   "metadata": {},
   "source": [
    "传回的是一个稀疏矩阵，需要转换成矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "41bc4b1d-0d9e-4236-b915-89dbf2f4bee8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:16.387527Z",
     "iopub.status.busy": "2023-05-14T17:58:16.386603Z",
     "iopub.status.idle": "2023-05-14T17:58:16.395606Z",
     "shell.execute_reply": "2023-05-14T17:58:16.393647Z",
     "shell.execute_reply.started": "2023-05-14T17:58:16.387429Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "city_encoded = city_encoded.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f23a3d9c-0c64-4fdc-924f-59df5a99bae5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:17.454948Z",
     "iopub.status.busy": "2023-05-14T17:58:17.454244Z",
     "iopub.status.idle": "2023-05-14T17:58:17.467024Z",
     "shell.execute_reply": "2023-05-14T17:58:17.465714Z",
     "shell.execute_reply.started": "2023-05-14T17:58:17.454885Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.],\n",
       "       [1., 0., 0.]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_encoded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5f4f403d-3a49-4487-9ab9-d0585d53af81",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:19.581160Z",
     "iopub.status.busy": "2023-05-14T17:58:19.579657Z",
     "iopub.status.idle": "2023-05-14T17:58:19.590211Z",
     "shell.execute_reply": "2023-05-14T17:58:19.588978Z",
     "shell.execute_reply.started": "2023-05-14T17:58:19.581094Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "data[[\"CityA\", \"CityB\", \"CityC\"]] = city_encoded\n",
    "\n",
    "data = data.drop([\"City\", \"CityC\"], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cddd2cd9-eb4e-46fc-ae83-7c4baf00d262",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:20.043681Z",
     "iopub.status.busy": "2023-05-14T17:58:20.042973Z",
     "iopub.status.idle": "2023-05-14T17:58:20.076099Z",
     "shell.execute_reply": "2023-05-14T17:58:20.074871Z",
     "shell.execute_reply.started": "2023-05-14T17:58:20.043619Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>EducationLevel</th>\n",
       "      <th>Salary</th>\n",
       "      <th>CityA</th>\n",
       "      <th>CityB</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "      <td>45.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>7.8</td>\n",
       "      <td>2</td>\n",
       "      <td>80.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0</td>\n",
       "      <td>25.2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>5.1</td>\n",
       "      <td>0</td>\n",
       "      <td>30.4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2</td>\n",
       "      <td>65.7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>1.2</td>\n",
       "      <td>2</td>\n",
       "      <td>60.8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>8.6</td>\n",
       "      <td>1</td>\n",
       "      <td>50.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>6.9</td>\n",
       "      <td>2</td>\n",
       "      <td>70.3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>4.2</td>\n",
       "      <td>1</td>\n",
       "      <td>40.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>2.4</td>\n",
       "      <td>0</td>\n",
       "      <td>28.1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>5.3</td>\n",
       "      <td>0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>3.9</td>\n",
       "      <td>1</td>\n",
       "      <td>48.3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>7.2</td>\n",
       "      <td>2</td>\n",
       "      <td>84.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2</td>\n",
       "      <td>70.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>1.5</td>\n",
       "      <td>2</td>\n",
       "      <td>62.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>8.4</td>\n",
       "      <td>1</td>\n",
       "      <td>55.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>6.5</td>\n",
       "      <td>2</td>\n",
       "      <td>72.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>4.8</td>\n",
       "      <td>1</td>\n",
       "      <td>42.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>2.1</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>5.2</td>\n",
       "      <td>0</td>\n",
       "      <td>31.6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>20</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1</td>\n",
       "      <td>43.8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>7.6</td>\n",
       "      <td>2</td>\n",
       "      <td>78.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>22</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2</td>\n",
       "      <td>63.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>23</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>24</td>\n",
       "      <td>8.2</td>\n",
       "      <td>1</td>\n",
       "      <td>48.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>25</td>\n",
       "      <td>6.3</td>\n",
       "      <td>2</td>\n",
       "      <td>68.4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>26</td>\n",
       "      <td>4.6</td>\n",
       "      <td>1</td>\n",
       "      <td>38.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>27</td>\n",
       "      <td>2.5</td>\n",
       "      <td>0</td>\n",
       "      <td>24.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>28</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0</td>\n",
       "      <td>29.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>29</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1</td>\n",
       "      <td>41.6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>30</td>\n",
       "      <td>7.4</td>\n",
       "      <td>2</td>\n",
       "      <td>76.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>31</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2</td>\n",
       "      <td>60.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>32</td>\n",
       "      <td>1.7</td>\n",
       "      <td>2</td>\n",
       "      <td>56.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>33</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1</td>\n",
       "      <td>45.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>34</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2</td>\n",
       "      <td>65.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>35</td>\n",
       "      <td>4.3</td>\n",
       "      <td>1</td>\n",
       "      <td>36.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Unnamed: 0  YearsExperience  EducationLevel  Salary  CityA  CityB\n",
       "0            0              3.0               1    45.9    1.0    0.0\n",
       "1            1              7.8               2    80.5    0.0    0.0\n",
       "2            2              2.3               0    25.2    1.0    0.0\n",
       "3            3              5.1               0    30.4    1.0    0.0\n",
       "4            4             10.0               2    65.7    0.0    1.0\n",
       "5            5              1.2               2    60.8    0.0    0.0\n",
       "6            6              8.6               1    50.1    0.0    0.0\n",
       "7            7              6.9               2    70.3    1.0    0.0\n",
       "8            8              4.2               1    40.7    1.0    0.0\n",
       "9            9              2.4               0    28.1    1.0    0.0\n",
       "10          10              5.3               0    32.0    1.0    0.0\n",
       "11          11              3.9               1    48.3    1.0    0.0\n",
       "12          12              7.2               2    84.2    0.0    0.0\n",
       "13          13             10.0               2    70.9    0.0    1.0\n",
       "14          14              1.5               2    62.4    0.0    0.0\n",
       "15          15              8.4               1    55.6    0.0    0.0\n",
       "16          16              6.5               2    72.7    1.0    0.0\n",
       "17          17              4.8               1    42.9    1.0    0.0\n",
       "18          18              2.1               0    26.5    1.0    0.0\n",
       "19          19              5.2               0    31.6    1.0    0.0\n",
       "20          20              3.5               1    43.8    1.0    0.0\n",
       "21          21              7.6               2    78.3    0.0    0.0\n",
       "22          22             10.0               2    63.6    0.0    1.0\n",
       "23          23              1.8               2    58.0    0.0    0.0\n",
       "24          24              8.2               1    48.3    0.0    0.0\n",
       "25          25              6.3               2    68.4    1.0    0.0\n",
       "26          26              4.6               1    38.5    1.0    0.0\n",
       "27          27              2.5               0    24.9    1.0    0.0\n",
       "28          28              5.0               0    29.7    1.0    0.0\n",
       "29          29              3.6               1    41.6    1.0    0.0\n",
       "30          30              7.4               2    76.9    0.0    0.0\n",
       "31          31             10.0               2    60.2    0.0    1.0\n",
       "32          32              1.7               2    56.3    0.0    0.0\n",
       "33          33              8.0               1    45.2    0.0    0.0\n",
       "34          34              6.1               2    65.9    1.0    0.0\n",
       "35          35              4.3               1    36.7    1.0    0.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de55cbf4-3074-4fd2-aa7e-e140369ec39e",
   "metadata": {},
   "source": [
    "## 资料预处理-划分训练集、测试集\n",
    "训练集/测试集可以8/2这样分开。这里使用sklearn里的工具进行分割\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "57018632-c768-4c2b-9fb6-58814e71f558",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:28.719894Z",
     "iopub.status.busy": "2023-05-14T17:58:28.719137Z",
     "iopub.status.idle": "2023-05-14T17:58:28.730611Z",
     "shell.execute_reply": "2023-05-14T17:58:28.729321Z",
     "shell.execute_reply.started": "2023-05-14T17:58:28.719829Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "x = data[[\"YearsExperience\", \"EducationLevel\", \"CityA\", \"CityB\"]]\n",
    "y = data[\"Salary\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4a77beb2-9e78-4958-9be3-367543afd6c7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:29.496608Z",
     "iopub.status.busy": "2023-05-14T17:58:29.495911Z",
     "iopub.status.idle": "2023-05-14T17:58:29.530691Z",
     "shell.execute_reply": "2023-05-14T17:58:29.529143Z",
     "shell.execute_reply.started": "2023-05-14T17:58:29.496547Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=87)\n",
    "# random_state是为了生成的随机数与教程视频中一致。如果不设置，就会每次都随机划分集合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8de7cb17-0328-4cf3-a01a-8210c39901d4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:30.255827Z",
     "iopub.status.busy": "2023-05-14T17:58:30.255061Z",
     "iopub.status.idle": "2023-05-14T17:58:30.275586Z",
     "shell.execute_reply": "2023-05-14T17:58:30.274760Z",
     "shell.execute_reply.started": "2023-05-14T17:58:30.255763Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>EducationLevel</th>\n",
       "      <th>CityA</th>\n",
       "      <th>CityB</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>3.5</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>6.5</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1.2</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>2.5</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>1.7</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.3</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.8</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>6.9</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    YearsExperience  EducationLevel  CityA  CityB\n",
       "20              3.5               1    1.0    0.0\n",
       "16              6.5               2    1.0    0.0\n",
       "5               1.2               2    0.0    0.0\n",
       "27              2.5               0    1.0    0.0\n",
       "32              1.7               2    0.0    0.0\n",
       "2               2.3               0    1.0    0.0\n",
       "1               7.8               2    0.0    0.0\n",
       "7               6.9               2    1.0    0.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5f7c63b0-fde7-4616-b06f-a1fde354f0af",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:32.689270Z",
     "iopub.status.busy": "2023-05-14T17:58:32.688564Z",
     "iopub.status.idle": "2023-05-14T17:58:32.703275Z",
     "shell.execute_reply": "2023-05-14T17:58:32.702184Z",
     "shell.execute_reply.started": "2023-05-14T17:58:32.689209Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "26    38.5\n",
       "35    36.7\n",
       "34    65.9\n",
       "29    41.6\n",
       "25    68.4\n",
       "17    42.9\n",
       "12    84.2\n",
       "4     65.7\n",
       "23    58.0\n",
       "18    26.5\n",
       "11    48.3\n",
       "21    78.3\n",
       "22    63.6\n",
       "24    48.3\n",
       "3     30.4\n",
       "31    60.2\n",
       "8     40.7\n",
       "10    32.0\n",
       "14    62.4\n",
       "30    76.9\n",
       "28    29.7\n",
       "9     28.1\n",
       "0     45.9\n",
       "19    31.6\n",
       "33    45.2\n",
       "6     50.1\n",
       "15    55.6\n",
       "13    70.9\n",
       "Name: Salary, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "f0ed3085-c1cb-46b3-8111-39a7b526c06e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:42.745667Z",
     "iopub.status.busy": "2023-05-14T17:58:42.744897Z",
     "iopub.status.idle": "2023-05-14T17:58:42.753311Z",
     "shell.execute_reply": "2023-05-14T17:58:42.751698Z",
     "shell.execute_reply.started": "2023-05-14T17:58:42.745615Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 为了后续计算方便，这里都统一为numpy格式。\n",
    "x_train = x_train.to_numpy()\n",
    "x_test = x_test.to_numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d556ff7-6e07-4896-8b0d-0f7746432de1",
   "metadata": {},
   "source": [
    "此时的模型为：\n",
    "\n",
    "$$y=w1*年资 + w2*学历 + w3*CityA + w4*CityB +b$$\n",
    "\n",
    "即:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "170edaa8-1f30-405b-a372-8f4c8e566b42",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:44.422976Z",
     "iopub.status.busy": "2023-05-14T17:58:44.422250Z",
     "iopub.status.idle": "2023-05-14T17:58:44.433178Z",
     "shell.execute_reply": "2023-05-14T17:58:44.432065Z",
     "shell.execute_reply.started": "2023-05-14T17:58:44.422913Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9.6,  9.3, 13.1,  8.6, 13.3,  9.8, 11.2, 18. ,  5.8,  5.1,  8.9,\n",
       "       11.6, 18. , 10.2,  8.1, 18. ,  9.2,  8.3,  5.5, 11.4,  8. ,  5.4,\n",
       "        8. ,  8.2, 10. , 10.6, 10.4, 18. ])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "w = np.array([1, 2, 3, 4]) # 假设w1~4分别为1，2，3，4\n",
    "b = 0\n",
    "(x_train*w).sum(axis=1)  + b #每行加和再加b"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35c4fbc2-abe5-49f8-a524-08f76fa5b044",
   "metadata": {},
   "source": [
    "也就是说，我们的模型代码为：\n",
    "```\n",
    "y_pred = (x_train*w).sum(axis = 1) + b\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "28ca9931-d6ec-4639-843e-f6acf1bc8beb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:46.981832Z",
     "iopub.status.busy": "2023-05-14T17:58:46.981124Z",
     "iopub.status.idle": "2023-05-14T17:58:46.989531Z",
     "shell.execute_reply": "2023-05-14T17:58:46.988351Z",
     "shell.execute_reply.started": "2023-05-14T17:58:46.981769Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_pred = (x_train*w).sum(axis = 1) + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "0fa2aa54-0dc2-4451-b049-21e98fcb6666",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:58:51.553914Z",
     "iopub.status.busy": "2023-05-14T17:58:51.553181Z",
     "iopub.status.idle": "2023-05-14T17:58:51.563495Z",
     "shell.execute_reply": "2023-05-14T17:58:51.561938Z",
     "shell.execute_reply.started": "2023-05-14T17:58:51.553850Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9.6,  9.3, 13.1,  8.6, 13.3,  9.8, 11.2, 18. ,  5.8,  5.1,  8.9,\n",
       "       11.6, 18. , 10.2,  8.1, 18. ,  9.2,  8.3,  5.5, 11.4,  8. ,  5.4,\n",
       "        8. ,  8.2, 10. , 10.6, 10.4, 18. ])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "182e61f1-af5d-4e5a-b247-6f32dbc57b95",
   "metadata": {},
   "source": [
    "## cost function\n",
    "与简单的线性回归一样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "c6b6cbc4-49ce-48ca-a842-ed8efc7e3a9c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:59:06.135435Z",
     "iopub.status.busy": "2023-05-14T17:59:06.134885Z",
     "iopub.status.idle": "2023-05-14T17:59:06.142781Z",
     "shell.execute_reply": "2023-05-14T17:59:06.141475Z",
     "shell.execute_reply.started": "2023-05-14T17:59:06.135395Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def cal_cost(x_train, y_train, w, b):\n",
    "    y_pred = (x_train*w).sum(axis = 1) + b\n",
    "    cost = ((y_pred - y_train)**2).mean()\n",
    "    return cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "83e51c1b-4d0d-49fa-b02a-8b5d5ff021bc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T17:59:06.765841Z",
     "iopub.status.busy": "2023-05-14T17:59:06.765140Z",
     "iopub.status.idle": "2023-05-14T17:59:06.775899Z",
     "shell.execute_reply": "2023-05-14T17:59:06.775018Z",
     "shell.execute_reply.started": "2023-05-14T17:59:06.765778Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1853.0200000000002"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cal_cost(x_train, y_train, w, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e35c5c39-a413-42f9-a7b4-922e7b3db1ac",
   "metadata": {},
   "source": [
    "## 设置优化器-梯度下降\n",
    "根据斜率改变参数\n",
    "\n",
    "设定 cost function：\n",
    "$$cost = (真实数据 - 预测数据)^2$$\n",
    "\n",
    "梯度下降的方式为：\n",
    "\n",
    "$$w_{new} = w-w方向斜率*学习率$$\n",
    "\n",
    "那么，斜率的应该为：\n",
    "$$\n",
    "cost = (真实数据 - 预测数据)^2\n",
    "$$\n",
    "\n",
    "$$= (y - y_{pred})^2$$\n",
    "\n",
    "$$= (y - (w_1 * x_1 + w_2 * x_2 + w_3 * x_3 + b))^2$$\n",
    "\n",
    "$$= (y - w_1 * x_1 - w_2 * x_2 - w_3 * x_3 - b)^2$$\n",
    "\n",
    "令$k=y- w_2 * x_2 - w_3 * x_3 - b$\n",
    "\n",
    "$$=(k - w_1 * x_1)^2$$\n",
    "\n",
    "$$=k^2 -2kx_1w_1 + w_1^2x_1^2$$\n",
    "\n",
    "那么，求w1方向的偏微分。\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9236b2c-a66b-4a01-9480-7c49b2d7c811",
   "metadata": {},
   "source": [
    "$$w_{1-gradient} = -2kx_1 + 2x_1^2w_1$$ \n",
    "$$=2x_1(w_1x_1 - k))$$\n",
    "$$=2x_1(y_{pred} - y)$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b93c3c2c-1964-4048-8453-26e0348852a6",
   "metadata": {},
   "source": [
    "同理可得，各个方向上的斜率分别为\n",
    "$$w_{gradient} = 2x_i(y_{pred} - y)$$\n",
    "$$b_{gradient} = 2(y_{pred} -y)$$\n",
    "\n",
    "\n",
    "因此，我们的优化器可以如下设置：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "4379657a-d1af-4487-82dd-b7f399dd1937",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T18:47:13.367648Z",
     "iopub.status.busy": "2023-05-14T18:47:13.366880Z",
     "iopub.status.idle": "2023-05-14T18:47:13.378117Z",
     "shell.execute_reply": "2023-05-14T18:47:13.376633Z",
     "shell.execute_reply.started": "2023-05-14T18:47:13.367583Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "\n",
    "# \n",
    "def linear_optimize(x_train, y_train, w, b):\n",
    "    \"\"\"输入测试集，w集合，b，学习率，返回新的w集合，b\"\"\"\n",
    "    y_pred = (x_train*w).sum(axis=1) + b #注意，此处不能(x_train*w + b）.sum(axis=1),若这样的话，相当于b会被加多次。但是模型中我们值加一次。 \n",
    "    b_gradient = (y_pred - y_train).mean()\n",
    "    shape = x_train.shape[1]\n",
    "    w_gradient = np.zeros(shape)\n",
    "    for i in range(shape):\n",
    "        w_gradient[i] = (x_train[:,i]*(y_pred - y_train)).mean()\n",
    "        \n",
    "    # b_new = b - b_gradient*learning_rate\n",
    "    # w_new = w - w_gradient*learning_rate\n",
    "   \n",
    "    return w_gradient, b_gradient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "9c0a5f5e-a1f7-427b-9494-be112b2d214f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T18:47:14.755985Z",
     "iopub.status.busy": "2023-05-14T18:47:14.755241Z",
     "iopub.status.idle": "2023-05-14T18:47:14.769550Z",
     "shell.execute_reply": "2023-05-14T18:47:14.768299Z",
     "shell.execute_reply.started": "2023-05-14T18:47:14.755920Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-295.8       ,  -65.67142857,  -19.82857143,   -8.15714286]),\n",
       " -46.949999999999996)"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w = np.array([0,2,2,4])\n",
    "b = 0\n",
    "linear_optimize(x_train, y_train, w, b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "839eac8e-1b2b-44cc-afc8-171a373ce776",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T18:47:16.066254Z",
     "iopub.status.busy": "2023-05-14T18:47:16.065557Z",
     "iopub.status.idle": "2023-05-14T18:47:16.079058Z",
     "shell.execute_reply": "2023-05-14T18:47:16.078218Z",
     "shell.execute_reply.started": "2023-05-14T18:47:16.066192Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-248.62428571,  -56.72857143,  -16.93928571,   -6.58571429]),\n",
       " -40.07142857142858)"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w = np.array([1,2,2,4])\n",
    "b = 1\n",
    "linear_optimize(x_train, y_train, w, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f028dd5-0d73-4d5a-b705-9e4bc35a7f05",
   "metadata": {},
   "source": [
    "## 正式开始计算：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "9caeeb97-0e56-428d-988c-e076bc134ac6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:17:23.844923Z",
     "iopub.status.busy": "2023-05-14T19:17:23.844523Z",
     "iopub.status.idle": "2023-05-14T19:17:23.851942Z",
     "shell.execute_reply": "2023-05-14T19:17:23.850266Z",
     "shell.execute_reply.started": "2023-05-14T19:17:23.844890Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "w_init = np.array([1,2,2,4])\n",
    "b_init = 0\n",
    "learning_rate = 0.01\n",
    "run_iter = 30000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "id": "ca5b1b60-39a4-4da6-adbd-8a6bced6b0c5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:17:24.903403Z",
     "iopub.status.busy": "2023-05-14T19:17:24.902704Z",
     "iopub.status.idle": "2023-05-14T19:18:42.525410Z",
     "shell.execute_reply": "2023-05-14T19:18:42.524568Z",
     "shell.execute_reply.started": "2023-05-14T19:17:24.903340Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2c72ac8033a041668b5082fe0b38cb3c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/30000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "当前迭代次数    0,当前变化率为-500.9,成本为1.89e+03\n",
      "当前迭代次数 1000,当前变化率为-0.63,成本为33.9\n",
      "当前迭代次数 2000,当前变化率为-0.45678,成本为28.1\n",
      "当前迭代次数 3000,当前变化率为-0.39734,成本为26.4\n",
      "当前迭代次数 4000,当前变化率为-0.32626,成本为25.8\n",
      "当前迭代次数 5000,当前变化率为-0.25954,成本为25.5\n",
      "当前迭代次数 6000,当前变化率为-0.20282,成本为25.4\n",
      "当前迭代次数 7000,当前变化率为-0.15685,成本为25.3\n",
      "当前迭代次数 8000,当前变化率为-0.12054,成本为25.3\n",
      "当前迭代次数 9000,当前变化率为-0.092289,成本为25.3\n",
      "当前迭代次数10000,当前变化率为-0.070494,成本为25.2\n",
      "当前迭代次数11000,当前变化率为-0.05377,成本为25.2\n",
      "当前迭代次数12000,当前变化率为-0.040978,成本为25.2\n",
      "当前迭代次数13000,当前变化率为-0.031212,成本为25.2\n",
      "当前迭代次数14000,当前变化率为-0.023766,成本为25.2\n",
      "当前迭代次数15000,当前变化率为-0.018092,成本为25.2\n",
      "当前迭代次数16000,当前变化率为-0.013771,成本为25.2\n",
      "当前迭代次数17000,当前变化率为-0.010481,成本为25.2\n",
      "当前迭代次数18000,当前变化率为-0.0079769,成本为25.2\n",
      "当前迭代次数19000,当前变化率为-0.0060708,成本为25.2\n",
      "当前迭代次数20000,当前变化率为-0.0046201,成本为25.2\n",
      "当前迭代次数21000,当前变化率为-0.003516,成本为25.2\n",
      "当前迭代次数22000,当前变化率为-0.0026757,成本为25.2\n",
      "当前迭代次数23000,当前变化率为-0.0020363,成本为25.2\n",
      "当前迭代次数24000,当前变化率为-0.0015496,成本为25.2\n",
      "当前迭代次数25000,当前变化率为-0.0011793,成本为25.2\n",
      "当前迭代次数26000,当前变化率为-0.00089744,成本为25.2\n",
      "当前迭代次数27000,当前变化率为-0.00068296,成本为25.2\n",
      "当前迭代次数28000,当前变化率为-0.00051974,成本为25.2\n",
      "当前迭代次数29000,当前变化率为-0.00039553,成本为25.2\n"
     ]
    }
   ],
   "source": [
    "c_hist = []\n",
    "w_hist = []\n",
    "b_hist = []\n",
    "w = w_init\n",
    "b = b_init\n",
    "\n",
    "progress = tqdm(range(run_iter))\n",
    "for i in progress:\n",
    "    cost = cal_cost(x_train, y_train, w, b)\n",
    "    c_hist.append(cost)\n",
    "    w_hist.append(w)\n",
    "    b_hist.append(b)\n",
    "    \n",
    "    # 更新w，b\n",
    "    w_gradient, b_gradient = linear_optimize(x_train, y_train, w, b)\n",
    "    w = w - w_gradient*learning_rate\n",
    "    b = b - b_gradient*learning_rate\n",
    "    \n",
    "    changing = (w_gradient + b_gradient).sum()\n",
    "    \n",
    "    if i%100 == 0:\n",
    "        progress.set_description(f\"当前迭代次数 {i:5},当前变化率为 {changing:4.5},成本为 {cost:4.3}\")\n",
    "    if i%1000==0:\n",
    "        print(f\"当前迭代次数{i:5},当前变化率为{changing:4.5},成本为{cost:4.3}\")\n",
    "    \n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "id": "4532ed7d-29c0-4c2f-9054-1a25beca63cd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:20:49.610821Z",
     "iopub.status.busy": "2023-05-14T19:20:49.610078Z",
     "iopub.status.idle": "2023-05-14T19:20:49.621702Z",
     "shell.execute_reply": "2023-05-14T19:20:49.620113Z",
     "shell.execute_reply.started": "2023-05-14T19:20:49.610758Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[28.30959489, 41.40287087, 19.83503077, 22.75620678],\n",
       "       [33.06964185, 60.04953496, 19.83503077, 22.75620678],\n",
       "       [24.66022556, 60.04953496, 22.75620678, 22.75620678],\n",
       "       [26.72291257, 22.75620678, 19.83503077, 22.75620678],\n",
       "       [25.45356672, 60.04953496, 22.75620678, 22.75620678],\n",
       "       [26.40557611, 22.75620678, 19.83503077, 22.75620678],\n",
       "       [35.13232886, 60.04953496, 22.75620678, 22.75620678],\n",
       "       [33.70431477, 60.04953496, 19.83503077, 22.75620678]])"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test*w + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "id": "b90cd5e2-32b4-4ed0-9d0c-fbe2050901ac",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:20:49.961692Z",
     "iopub.status.busy": "2023-05-14T19:20:49.960992Z",
     "iopub.status.idle": "2023-05-14T19:20:49.972726Z",
     "shell.execute_reply": "2023-05-14T19:20:49.971009Z",
     "shell.execute_reply.started": "2023-05-14T19:20:49.961629Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1.58668232,  18.64666409,  -2.92117601, -10.8165801 ])"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "id": "8acaff08-95f1-4264-a7c6-0900bd58de0a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:20:50.405634Z",
     "iopub.status.busy": "2023-05-14T19:20:50.404963Z",
     "iopub.status.idle": "2023-05-14T19:20:50.414557Z",
     "shell.execute_reply": "2023-05-14T19:20:50.412958Z",
     "shell.execute_reply.started": "2023-05-14T19:20:50.405572Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "22.756206775811595"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "id": "281ac241-e3b2-4b7e-9231-f4b92099f8dd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:20:50.922640Z",
     "iopub.status.busy": "2023-05-14T19:20:50.921964Z",
     "iopub.status.idle": "2023-05-14T19:20:50.932942Z",
     "shell.execute_reply": "2023-05-14T19:20:50.932028Z",
     "shell.execute_reply.started": "2023-05-14T19:20:50.922578Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3.5, 1. , 1. , 0. ],\n",
       "       [6.5, 2. , 1. , 0. ],\n",
       "       [1.2, 2. , 0. , 0. ],\n",
       "       [2.5, 0. , 1. , 0. ],\n",
       "       [1.7, 2. , 0. , 0. ],\n",
       "       [2.3, 0. , 1. , 0. ],\n",
       "       [7.8, 2. , 0. , 0. ],\n",
       "       [6.9, 2. , 1. , 0. ]])"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1c1868b-5f5c-46b6-a0a4-467cb52f9116",
   "metadata": {},
   "source": [
    "## 使用测试集验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "id": "a09b6eda-f78e-4cf6-b398-99025663774f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:20:51.417613Z",
     "iopub.status.busy": "2023-05-14T19:20:51.416950Z",
     "iopub.status.idle": "2023-05-14T19:20:51.426396Z",
     "shell.execute_reply": "2023-05-14T19:20:51.425038Z",
     "shell.execute_reply.started": "2023-05-14T19:20:51.417551Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "prediction = pd.DataFrame((w*x_test).sum(axis = 1) + b, columns=[\"Prediction\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "id": "10350cb7-3949-4ffa-a4d8-fcaf72f584b6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-14T19:20:52.457864Z",
     "iopub.status.busy": "2023-05-14T19:20:52.457187Z",
     "iopub.status.idle": "2023-05-14T19:20:52.473628Z",
     "shell.execute_reply": "2023-05-14T19:20:52.472520Z",
     "shell.execute_reply.started": "2023-05-14T19:20:52.457800Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Salary</th>\n",
       "      <th>Prediction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>43.8</td>\n",
       "      <td>44.035083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16</td>\n",
       "      <td>72.7</td>\n",
       "      <td>67.441794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>60.8</td>\n",
       "      <td>61.953554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>27</td>\n",
       "      <td>24.9</td>\n",
       "      <td>23.801737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>32</td>\n",
       "      <td>56.3</td>\n",
       "      <td>62.746895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>25.2</td>\n",
       "      <td>23.484400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>80.5</td>\n",
       "      <td>72.425657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>70.3</td>\n",
       "      <td>68.076467</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  Salary  Prediction\n",
       "0     20    43.8   44.035083\n",
       "1     16    72.7   67.441794\n",
       "2      5    60.8   61.953554\n",
       "3     27    24.9   23.801737\n",
       "4     32    56.3   62.746895\n",
       "5      2    25.2   23.484400\n",
       "6      1    80.5   72.425657\n",
       "7      7    70.3   68.076467"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat([real_data.reset_index(), prediction],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f1212c2-a8bf-4bc8-b4df-46c55f3c5c4d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-15T07:48:05.658071Z",
     "iopub.status.busy": "2023-05-15T07:48:05.657331Z",
     "iopub.status.idle": "2023-05-15T07:48:05.666499Z",
     "shell.execute_reply": "2023-05-15T07:48:05.664716Z",
     "shell.execute_reply.started": "2023-05-15T07:48:05.658006Z"
    }
   },
   "source": [
    "除了直观地查看预测与实际之外，还应当再计算一次成本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "id": "5b5f6636-e87a-48a4-9bc6-57799e4e0d8f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-05-15T07:48:28.330360Z",
     "iopub.status.busy": "2023-05-15T07:48:28.329626Z",
     "iopub.status.idle": "2023-05-15T07:48:28.341524Z",
     "shell.execute_reply": "2023-05-15T07:48:28.340318Z",
     "shell.execute_reply.started": "2023-05-15T07:48:28.330296Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18.11071413992474"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cal_cost(x_test, y_test, w, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc3f7b8-796c-4e8f-b09f-5a5c978905b5",
   "metadata": {},
   "source": [
    "在测试集中，成本为18，与训练集差别不大。所以可以认为本次预测比较准。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f034b31a-55b4-43c3-adc2-89698671c911",
   "metadata": {},
   "source": [
    "------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a27cf61b-d161-4f9b-b3a2-e59f750fdce2",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
